______________________________________ U.S. Patents U.S. Pat. No. Inventor Issue Date ______________________________________ 3,116,555 Helava 01/1964 5,309,522 Dye 05/1994 ______________________________________ Publications Manual of Photogrammetry, 4th Ed. 1980, published by the American Society of Photogrammetry
Presently, topographic maps that contain terrain elevation information are produced using photogrammetric instruments such as stereoplotters. There are three types of stereoplotters: (1) analog stereoplotters, (2) analytic stereoplotters, and (3) softcopy stereoplotters. Softcopy stereoplotters are more capable of automated processing than analytic stereoplotters through extensive use of modern computer technology and digital image processing methods. Analytic stereoplotters are, in turn, more capable of automated processing than analog stereoplotters because of the former's use of computerized method to implement mathematical models to represent the stereo projection model. The discussion of the background will focus mostly on analytic stereoplotters and softcopy stereoplotters.
Stereoplotters utilize a pair of partially overlapping photographs, which are usually taken from an aircraft flying over an area of mapping interest, to extract topographic elevation information. Some of the characteristics of each photograph are known prior to elevation extraction, some are supplied through a process of aerotriangulation or by other means. The characteristics of the photographs that are important to elevation extraction include the three dimensional coordinates of the camera stations, the roll, pitch, yaw angles of the camera at the instant of each camera exposure, the interior orientation of camera, and a flight bearing angle of the aircraft. In stereoplotters, a stereo pair of photographs are utilized to construct a three dimensional (3D) model through stereoscopic viewing means to represent topographic surface of the area of mapping interest. When the stereo pair is viewed through stereoviewing instrument, the elevation of an image feature can be determined by measuring a stereo parallax of the image feature.
According to chapter 13 of "Manual of Photogrammetry", to measure the stereo parallax of an image feature in a stereo image pair using a stereoplotter or stereocomparator, an operator is required to perform at least three types of activities. These activities include:
a) setting up a relative orientation and an absolute orientation of a stereo model, which is a three-dimensional model formed by intersecting rays of an overlapping pair of photographs, PA1 b) specifying formula for correction and adjustment, PA1 c) collecting and editing elevation data for each image feature that is of interest. PA1 a) preparing diapositives of photograph from film negatives and loading diapositives, PA1 b) Identification of image feature, PA1 c) Stereo model setup, PA1 d) parallax measurement and data collection. PA1 a) Establishing an equally-spaced grid, on which digital elevation data is to be extracted, on one photograph of the stereo pair. PA1 b) At each grid crossing, an image patch of suitable size is selected. PA1 c) Using pattern matching techniques that is well known in the art, a corresponding image patch in the conjugate photograph is identified. PA1 d) The corresponding photograph coordinate on the conjugate photograph of the grid crossing is then obtained from correlation methods that is well known in the art. This allows the determination of a parallax for the grid crossing in question. PA1 e) The elevation at the grid crossing in question is determined from the x-parallax after the stereo model is set up by eliminating the y-parallax. PA1 f) Repeating the above recipe from b) through e) on the rest of grid crossings results in a rasterized digital elevation model that can be generated on a computer autonomously. PA1 a) The degree of success on using pattern matching techniques to identify similar image patches in a stereo image pair is influenced by many factors, such as image contrast, and image texture. It is easy to see that when the image patch does not contain objects of significant contrast or gradient variations, the results of pattern matching will be ambiguous. PA1 b) Poor image contrast or gradient variation will also leads to greater errors in the results of correlation calculation of photograph coordinates. PA1 c) Since image features or prominent terrain features that are of interest to map production can not be expected to fall on a regular grid pattern, rasterized digital elevation model can miss important topographic features such as break lines or isolated point features, unless the grid density is increased sufficiently. However, increasing the grid density in generating digital elevation data will lead to inefficiency and geometrically increasing computing requirement. PA1 (a) retrieving a pair of aerial images (stereo image pair) that covers an overlapping, sufficiently large area of a terrestrial region, but are taken at two different camera stations, into a random access memory device, PA1 (b) retrieving two sets of airborne control data, each of which includes a camera station, a set of camera parameters, a roll angle, a pitch angle, a yaw angle, a flight bearing angle, for said pair of aerial images into said random access memory device, PA1 (c) extracting point and edge features from said stereo image pair using edge enhancing band-pass filters, PA1 (d) generating a list of common feature points from the overlapping region of said edge-enhanced stereo image pair using correlation techniques, PA1 (e) associating each entry in said list of common feature points with two pair of photograph coordinates, a first pair of photograph coordinate is associated with the left image of said stereo image pair, and a second pair of photograph coordinate is associated with the right image of said stereo image pair, PA1 (f) calculating a three dimensional space coordinate for a feature point in said common list of feature points, projected on a horizontal projection plane at a height h above a known reference Datum plane using said first set of airborne control data associated with said left image, PA1 (g) calculating a second three dimensional space coordinate for said feature point in said common list of feature points, projected on said horizontal projection plane at said height h above said reference Datum plane using said second set of airborne control data associated with said right image, PA1 (h) evaluating an object-space parallax between said first space coordinate and said second space coordinate associated with said feature point in said common list of feature points, PA1 (i) estimating the true elevation h' using said object-space parallax according to a parallax formula relating a base length of said stereo image pair, flight altitude, and said object-space parallax, PA1 (j) calculating a revised space coordinate projected on a new projection plane at said new height h' above said reference Datum plane, PA1 (k) proceeding to the next feature point in said list of common feature list and repeating the steps (f) through (j). PA1 (1) The method and apparatus disclosed in the present invention is applicable to both digitized film image as well as digital raster image data from digital imaging sensors. PA1 (2) The present invention does not require any ground control data that are obtained by conducting a ground survey in order to perform photogrammetric processing. PA1 (3) The present invention does not require any stereo-viewing equipment or stereo viewing perception by an operator. PA1 (4) The present invention does not require setting up a stereo model by removing the y-parallax between a stereo image pair. PA1 (5) Digital elevation data can be extracted directly from un-corrected image or from mission medium containing unprocessed data. PA1 (6) The present invention extracts topographic elevation of all significant image features to improve the accuracy of autonomous generation of digital elevation data. PA1 (7) The cost of extracting elevation information from raw image data is greatly reduced by eliminating the need for ground survey, eliminating the use of stereo viewing devices.
In order to setup the proper orientation for a stereo model, a manual procedure, aided with the use of a stereo viewing instrument, is required to eliminate a y-axis component of the stereo parallax on at least five image points. The stereo parallax that is associated with an image feature that appears on two photographs taken at two different camera stations (or projection center) has an x-axis component (x-parallax) and a y-axis component (y-parallax) in general, where the x-axis is the direction of flight and the y-axis is perpendicular to the x-axis and parallel to the plane of the photograph. The y-axis component of a parallax needs to be eliminated so that a line in the stereo model that are established by two image points will remain in the same direction.
Specifying formulas for correction and adjustment allows stereoplotters to take into consideration various geometric distortions in the photographs that are caused by factors such as aberration of the camera lens, curvatures of the earth's surface, atmospheric effect, and other systematic errors.
To determine the elevation of an image feature after the stereo model is set up and the y-parallax is removed, stereoplotters employs a viewing aid, which is commonly referred to as a floating marker. The floating marker consists of a pair of movable dots, each of which is positioned on one of the stereo pair of photographs. When viewed through the stereoscopic viewing device, the floating marker provides to an operator an apparent vertical depth. The apparent vertical depth of the floating marker can be adjusted by the operator, so that under stereoscopic viewing, the floating marker may appear above, beneath, or coincident with the image feature of interest. Collection of a digital elevation database is accomplished by moving the floating marker to visit various image features in the overlapped region of the stereo pair, and collecting the apparent vertical depth of the floating marker at each image feature point when the floating marker coincides with the image feature under stereoscopic view.
As discussed above, the photogrammetric processing of stereo pair of photographs to generate digital elevation database is far from fully automated. The major deficiencies that are obstacles to fully automated operation are:
All four aspects of the stereoplotter operation requires manual intervention. U.S. Pat. No. 3,116,555 to Helava discloses an analytic stereoplotter that can produce digital elevation database, but suffers from its requirement to have a skilled operator to perform a) image point identification, b) stereo model setup, and c) parallax measurement and data collection. After elevation data collection is completed for each stereo model, new diapositives needs to loaded manually into the stereoplotter.
In softcopy stereoplotters, photographs are scanned into a digital format for computer processing, in contrast to analytic stereoplotters that use a transparent diapositive of the photograph. In digital format, each photograph is converted into an equally-spaced matrix array of intensity values (a raster image), and each pixel of the digital raster image is associated with a column and a row number measured from a center point or a principal point of the photograph. The principal point is the intersection of the optic axis of the camera with the film plane. The column and row coordinate measured on the plane of photograph is referred to as a photograph coordinate. Improvement in automation of extracting digital elevation databases have been made using digital image processing techniques including pattern matching and correlation. This is accomplished by:
In practice, automated extraction of raster digital elevation model by pattern matching and correlation techniques are far from achieving a satisfactory degree of accuracy when compared to results obtained by manual procedures practiced on analytic stereoplotters. Frequently, topographic features such as irregular break lines or isolated, sharp point features are not captured by this automated process. The reasons are:
U.S. Pat. No. 5,309,522 to Dye discloses a novel approach to determine terrain elevation stereoscopically by forming additional, multiple sets of lower-resolution image sets and extracting edge features from these multiple resolution sets of stereo image pairs. The elevation information is built, in stages, from each stereo image pair at a given resolution by calculating the stereo parallax between corresponding edge features. While Dye's method makes claim that it can automate the generation of digital elevation databases from aerial photo or satellite images, it presumes that both of the stereo images are geometrically corrected, as the accuracy of the geometric correction will determine the accuracy of the extracted elevation data. While geometric correction is performed by methods that is well known in the art to generate mapping polynomials for transforming each image, the process of performing geometric correction is still manual and labor intensive. In addition, carrying out the geometric correction process will require ground control data that need to be acquired either from ground survey, or digitizing existing maps. Thus, the scope of automation that can be accomplished by Dye's method is limited. Specifically, Dye's method will not be able to generate digital elevation databases from raw image data autonomously.
According to the parallax formula that Dye's methods employs to calculate elevation: EQU Z=ZL (X1-X)/(X1-XL),
where Z is the true elevation, ZL is the height of a camera station, X1 is a projected location of the pixel of interest from one image, X is the true projected location of the pixel of interest, and XL is the location of the nadir point of the image being projected. The value of X1, X, XL is accurate only if the image is geometrically corrected. Consequently, the accuracy of extracted elevation is questionable when the image is not properly corrected for geometric fidelity. The term (X1-XL) is only a fraction of the base distance (XR--XL) of the stereo pair of images, and the fraction value varies as X1 changes. Thus, the resolution of extracted elevation value is coarser than other methods that uses the base distance (XR--XL). Also, varying values of (X1--XL) implies that the accuracy of extracted elevation value will vary accordingly, producing less consistent digital elevation model. It is also apparent that if the images are not geo-referenced prior to being processed by Dye's method, the results of the elevation extraction is indexed only to photograph coordinates, but can not be indexed to geographic coordinates referenced to known Datum. Furthermore, the resolution of elevation determination is limited to the ground sampling distance divided by the partial base length (X1-XL) to height ratio. In addition, the generation of multiple resolution data sets implies that the amount of raster data to be processed will approach close to 1.5 times of the original raster image.
Thus it is recognized that previous methods and techniques disclosed in the art and practiced in stereoplotters in operation has accomplished automation only in individual tasks, manual operations are still required to proceed from one automated task to another task. Autonomous generation of digital elevation databases from raw image data is well beyond the degree of automation that has been accomplished in the art. The extraction of topographical elevation directly from mission medium containing unprocessed image data is greatly needed, but can not be accomplished by stereoplotters currently in use. Use of stereoscopic viewing device is a obstacle towards automated production of digital elevation database. The manual practice of setting up a stereo model by eliminating y-parallax inhibits autonomous operation of digital elevation data extraction.